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Attorney Docket No. 11423-002001 

METHOD OF REPLACING CONTENT 

TECHNICAL FIELD 

This invention relates to content replacement, and more particularly to the 
replacement of an advertisement with another advertisement, other content, or no content. 

5 BACKGROUND 

Referring to Fig. 1, a multimedia computer system 100 is illustrated which represents 
a typical hardware setup for executing software that allows a user to perform tasks such as 
communicating with other computer users, accessing various computer resources, and 
viewing, creating, or otherwise manipulating electronic content - that is, any combination of 

10 text, images, movies, music or other sounds, animations, three-dimensional virtual worlds, 

and links to other possibly distributed objects. The system 100 includes various input/output 
(I/O) devices (mouse 102, keyboard 104, display or television screen 106, remote controller 
108, printer 110) and a general purpose computer 1 12 having a central processor unit (CPU) 
1 14, an I/O unit 116 and a memory 118 that stores data and various programs such as an 

15 operating system 120, and one or more application programs 122. The various I/O devices 
may be connected to the computer 1 12 via a wireless link (as shown by the dashed line for 
the remote controller 108) or a direct link. 

The operating system 120 is a software program that manages the basic operations of 
a computer system. The operating system 120 determines how the computer main memory 

20 will be apportioned, how and in what order tasks assigned to it are handled, how it will 
manage the flow of information into and out of the main processor 114, how it will get 
material to a printer for printing, to the screen 106 for viewing, how it will receive 
information fi-om the keyboard 104. In addition, the operating system 120 may manage the 
transfer of multimedia information to display devices. In short, the operating system 120 

25 handles the computer's basic housekeeping. Examples of operating systems include MS- 
DOS, UNIX, PICK, MS-Windows, and Linux. 

An application program 122 or appUcation is a software program that carries out 
some useftil task. Examples of applications include database managers, spreadsheets, 
communications packages, graphics programs, word processors, and browsers. 
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The computer system 100 also typically includes some sort of communications card 
or device 124 (for example, a telephone, computer network, or cable line). A content 
provider provides access to network content in a network 126 in addition to various services 
associated with the content. Examples of content providers include AT&T WorldNet, 

5 UUNet, and America Online. 

Additionally, for multimedia computer systems, a user of the system 100 can watch 
television or view web pages and otherwise "surf' the Internet. In this regard, the computer 
1 12 might include a special purpose computer, referred to as a set top device that is used in 
connection with television sets for viewing web pages on the Internet. In such a special 

1 0 purpose computer, a television tuner 128 is included for receiving broadcast and/or cable 
television signals 130. The monitor in such a multimedia system might correspond to a 
computer monitor, a television monitor, or any combination of both. 

As shown in Fig. 2, a user of the computer system 100 can access electronic content 
or other resources either stored locally at the user's own cUent system 202 (for example, a 

15 personal or laptop computer) or remotely at one or more server systems 200. An example of 
a server system is a host computer that provides subscribers with online computer services 
such as, for example, e-mail, e-commerce, chat rooms, Internet access, electronic newspapers 
and magazines. Users of a host computer's online services typically communicate with one 
or more central server systems 200 through client software executing on their respective 

20 chent systems 202. This may or may not include proxy server functionality. Thus, the user 
of the cUent station 202 may connect to the server 200 via a traditional modem 204, a special 
integrated service digital network (ISDN) modem 206, or via a TCP/IP protocol 208. 

In practice, a server system 200 typically will not be a single monolithic entity but 
rather will be a network of interconnected server computers, possibly physically dispersed 

25 from each other, each dedicated to its own set of duties and/or to a particular geographical 
region. In such a case, the individual servers are interconnected by a network of 
commimication links, in known fashion. 

As mentioned above, one type of application program is a browser. A browser is an 
example of client software that enables users to access and view electronic content stored 

30 either locally or remotely, such as in a network environment (local area network (LAN), 

intranet, Internet). A browser typically is used for displaying documents described in Hyper- 
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Text Markup Language (HTML) and stored on servers connected to a network such as the 
Internet. 

A user instructs a browser to access an HTML document, or web page, by specifying 
a network address or Uniform Resource Locator (URL) -- at which a desired document 

5 resides. In response, the browser contacts the corresponding server hosting the requested 
web page, retrieves the one or more files that make up the web page, and then displays the 
web page in a window on the user's screen 106. 

Fig. 3 is a screen shot of a browser apphcation 300 (Internet Explorer) displaying a 
typical HTML document, or web page 302. As shown therein, a single web page may be 

10 composed of several different files potentially of different data types 304 (for example, text, 
images, virtual worlds, sounds, or movies). In addition, a web page can include links 306, or 
pointers, to other resources (for example, web pages, individual files, or broadcast television 
stations) available on the network or through the television signal 130. Each link has an 
associated URL pointing to a location on the network. When a user chcks on, or otherwise 

15 selects a displayed Unk, the browser automatically will retrieve the web page or other 

resource corresponding to the link's associated URL and display it to, or execute it for, the 
user. 

An advertisement 308 is displayed in the browser apphcation 300 for VISA. In this 
case, the originator of content (presumably VISA) displays an image of a VISA card (which 
20 resembles the VISA logo) along with a graph of annual percentage rates associated with 
credit cards. 

Content filtration systems may be used to manage and/or present content choices to 
the user. One example of such a system is a portal which convinces users to visit the portal 
web site and provides, at a minimum, search engines, fi^ee e-mail, instant messaging and chat, 

25 web hosting. Content filtration systems could provide a user with content from selected 

channels and sources by automatically delivering the content to the user's computer via the 
Internet at user-specified intervals (referred to as "push" delivery). Some content filtration 
systems prompt the user to actively rank content choices (both likes and dislikes). 

A content filtration system may monitor, block and even report content usage to the 

30 user or another user that supen^ises the user. The content filtration system might block or 
deny content that the supen^isory user or the user finds inappropriate, for example, vulgar 
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language, pornography, or hate Hterature. The content filtration system might provide a 
protective or defensive gateway to certain users -- for example, a firewall limits exposure of a 
computer or group of computers to an attack from outside. Firewalls are often used on a 
local area network (LAN) connected to the Internet. The firewall enforces a virtual boundary 
5 between two or more networks. 



SUMMARY 

Various implementations may include one or more of the following features. 

In one implementatioiij a computer-implemented method of controlling content 
10 includes receiving input from a user and monitoring content in an application program or 

content exchanged between the application program and an operating system residing at the 
user's computer system. The method includes determining whether monitored content 
corresponds to a predetermined advertisement. If the monitored content corresponds to the 
predetermined advertisement, the predetermined advertisement is replaced with another 
1 5 predetermined advertisement based on the user input. 

Because an advertisement is replaced based on user input, the method provides the 
user with a way of obtaining more useful and relevant information. For example, an 
advertisement for a remote company may be replaced with an advertisement for a company 
local to the user. Furthermore, entities related to the advertisements may agree upon the 
20 replacement technique and effectively act to co-brand each other. This provides one of the 
entities with the ability to reach out to users that wouldn't normally be reached. 

In another implementation, a method of filtering content relating to an originator of 
content includes receiving input from a user selecting an arbitrary originator of content and 
receiving a request from the user at a computer for an internetwork page. The method 
25 includes determining whether the internetwork page includes content relating to the 

originator. If the internetwork page includes content relating to the originator, the related 
originator content is replaced with other predetermined content. 

Because a user is able to select an arbitrary originator of content, the user is enabled 
to effectively "boycott" or ignore electronic content relating an originator of content, thus 
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ignoring all goods and services provided by the originator of content through the 
internetwork. 

The details of one or more embodiments of the invention are set forth in the accompa- 
nying drawings and the description below. Other features, objects, and advantages of the 
invention will be apparent from the description and drawings, and from the claims. 

DESCRIPTION OF DRAWINGS 

Fig. 1 is a block diagram of a multimedia computer system. 

Fig. 2 is a block diagram of a typical network computing environment. 

Fig. 3 is a screen shot of a browser displaying a web page. 

Fig. 4 is a block diagram of a content control system according to the invention. 

Figs. 5 and 12 are flow charts of procedures for recognizing and removing content in 
the content control system of Fig. 4. 

Fig. 6 is a flow chart of a procedure for setting user preferences in the content control 
system of Fig. 4. 

Fig. 7 is a screen shot of a browser displaying a preference window for setting user 
preferences in the content control system of Fig. 4. 

Fig. 8 is a flow chart of a procedure for activating a set of replacement rules in the 
content control system of Fig. 4. 

Fig. 9 is a block diagram of different types of designator strings for removed content. 

Fig. 10 is a flow chart of a procedure for inserting new content into a user-accessed 
document. 

Fig. 1 1 is a screen shot of a browser displaying inserted new content. 
Like reference symbols in the various drawings indicate like elements. 

DETAILED DESCRIPTION 

Referring to Fig. 4, a system 400 may be used to recognize data or content that a user 
might experience and replace the recognized data with substantive data, information, or 
content. This system 400 operates with the complete support, legal authority, and command 
of the user, based on the user's objectives. 
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In Fig. 4, an observation program 405 is implemented by and stored at a computer to 
observe events that occur in a computer system 410. The observation program 405 may 
reside at one or more locations in the computer system 410. For example, the observation 
program 405 may reside at a user's client computer 415. The user's chent computer 415 
5 includes any computer system such as the computer system of Fig. 1. Thus, the client 
computer 415 may be a desktop personal computer, a laptop computer, a networked 
computer, a cellular telephone, or a personal data/digital assistant. 

If the chent computer 415 is connected to a network, then the user's chent computer 
415 makes requests from a server computer 420 that provides services to one or more other 

10 computer programs 425, for example, name services, portal and apphcation servers, 

electronic mail, and Intemet service. The user can then access information or documents 
through the server computer 420 using, for example, a dial-in technique, a local area network 
(LAN), a cable TV modem, a networked computer, an internetwork TV hybrid connection, 
an online service provider, an ISDN line, SLIP or PPP modems, a dumb terminal, or a 

1 5 terminal emulation technique. For example, a web browser at the user's client computer 415 
might request HTML files from the network server computer 420. 

The observation program 405 may reside at a server computer 420. For example, the 
observation program 405 may reside at a proxy server that acts as an intermediary between 
the user's computer 415 and the Intemet so that the originator of information can ensure 

20 security, administrative control, and caching service to the user. As another example, the 
observation program 405 may reside at a firewall server that protects resources of a private 
network from other networks. For example, if the user's computer 415 is part of an intranet 
430 that allows its workers access to the wider Intemet, the intranet may use a firewall server 
that prevents outsiders from accessing its own private data resources and for controlling what 

25 outside resources its own users have access to. The observation program 405 may be 
distributed over more than one computer, or over a network. 

Examples of an observation program 405 include a kemel module, an application, a 
dynamic link library (dll), a dynamically loaded library, and/or an apphcation device driver. 
The observation program 405 may communicate with the computer system 410 and in 

30 particular, the user's computer 415 and the server computer 420, using mechanisms such as 
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application kernel calls, dynamic data exchange (DDE), direct RAM memory access, 
Windows tags, Windows event and messages. 

The user's computer 415 includes, among other features, an operating system 435 that 
manages the basic operations of the computer 415 and one or more local application 
5 programs 440 that perform some useful tasks. 

An example of how the system 400 might be implemented is now discussed. A user 
accessing a financial planner apphcation program residing at the user's computer 415 peraiits 
an observation program 405 to monitor information exchanged between the financial planner 
apphcation and the user's operating system 435. When the observation program 405 detects 

1 0 the presence of an advertisement for a first bank, the observation program 405 removes the 
advertisement (and therefore prevents it fi^om reaching the user) and replaces the 
advertisement for the first bank with an advertisement or service for a second bank (which 
presumably has more relevant or useful information for the user). In this situation, perhaps 
the first bank has an agreement with the financial planner that permits a financial planner 

15 application user to directly download information relating to the first bank's services and 

trades. However, the second bank, because it is too small or too local, may not have such an 
agreement with the financial planner. Thus, the user receives information about the second 
bank rather than the first bank. If the received information includes an advertisement, then 
the system effectively co-brands the first and second banks. 

20 Such a system saves the second bank money because it need not set up an agreement 

with the financial planner. The first bank might provide this service to one or more other 
banks and use this service to negotiate a better deal with the purveyor of the financial 
planner. The financial planner may benefit fi-om having fewer (but more lucrative) 
customers to interact with. 

25 In another implementation, the user might access a web site operated by the financial 

planner. In this case, the observation program 405 could monitor content at the web site and 
replace an advertisement relating to the first bank with an advertisement relating to the 
second bank. 

Referring also to Fig. 5, the observation program 405 performs a procedure 500 for 
30 recognizing content in the network computer system 410. Content to be recognized may be 
content exchanged between the operating system 435 and the local application program 440, 
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between the operating system and the server 420, between the server 420 and one or more 
remote applications 425, and between documents within the remote appUcations. Content to 
be recognized may be content accessed by the operating system 435, the apphcation program 
440, or the server 420. 

5 Content includes, for example, text, video, sounds, images, or movies. Furthermore, 

content may include information associated with an originator of content. For example, an 
advertisement or notice is content that is designed to attract pubhc attention or patronage. An 
advertisement generally calls public attention or patronage to a product, service, or business. 
An advertisement includes logos on apphcation programs as well as icons on a desktop, 

1 0 Advertisements also include text, text placed in a hyper-tag, or text replaced with a hyper- 
tag. 

The observation program 405 receives user preferences (step 505). Preferences from 
the user indicate some information about the content that the user wishes to control in 
addition to some information about the user. The observation program 405 may receive the 

15 content preferences at any time and from sources other than the user. For example, the user 
may enter content preferences during a special enrollment period or during regular operation 
of the user's computer system 415. Likewise, the user may enter preferences at any time 
during regular operation of the user's computer system 415 by opening a preference selection 
apphcation in the observation program. Preferences may be received by an organization 

20 separate from user input - thus, they may be updated at any time by the organization. 

Preferences may be generated automatically by the observation program or at another 
server by a preference server and then accessed by the observation program. When 
generating the preferences, the preference server could observe information relating to the 
user and generate the preferences based on these observations. For example, the preference 

25 server might generate a profile of the user that includes a location of the user, a time the user 
accesses content, or a history of user interests. The preference server may set up a preference 
for a bank that is physically nearer to the user. The idea here is that the user would be more 
likely to interact with content relating to a local bank than with content relating to a national 
or remote bank. Alternatively, many local banks may combine their resources to advertise 

30 only to those users determined to be in a same virtual (for example, based on IP address) or 
geographic region or place. 
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Referring also to Fig. 6, the user performs a procedure 600 for generating content 
preferences. The user may select content for the application program that is observed (step 
605) or the user may subscribe to an organization's or someone else's content preference 
(step 610). When the user selects the content, the user may specify the content (step 615) 

5 when wishing to block the content. The user could specify the originator of the content (step 
620) when wishing to block all content that originates from the originator of the content. 
When the appUcation program is a network browser, the user could specify the Internet 
protocol (IP) address that identifies a particular device from which the content originates and 
a particular network from which the content originates (step 625). The user specifies the IP 

1 0 address when wishing to block all content that originates from the same network or device. 
The user could specify a file address (for example, a uniform resource locator (URL) 
address) or some substring of the file address (step 630) when wishing to block all content 
that originates from the same file. 

Referring to Fig. 7, an example of how such a content selection is implemented is 

1 5 displayed in a network browser as a separate preferences window 700. When the user first 
observes a Company A advertisement 702, if the user wishes to control content relating to 
observed advertisement, the user selects the advertisement by, for example, meta-clicking 
(for example, alt-shift cHck) on the advertisement. The preference window 700 indicates to 
the user the different options for preferences relating to that content. The user may click on 

20 one or more of the appropriate boxes indicates such preferences ~ a box 705 selecting the 
URL or a substring of the URL address 705, a box 710 selecting the advertisement itself, a 
box 715 selecting all content originating from the originator of that advertisement, and a box 
720 selecting the IP address. 

Referring again to Fig. 6, the user may agree to accept content preferences indicated 

25 by another organization such a situation might occur with the financial planner observation 
program described above. Li that case, the user would agree to the first bank's content 
preferences. Then, the fu-st bank would implement the observation program at the will of the 
user. As a still further example, the other organization could be a special interest 
organization, such as, for example, an anti-smoking organization. The anti-smoking 

30 organization might have a list of content providers and/or content that it considers as being 
contrary to their set of beliefs. Thus, if the user agrees to accept the anti-smoking 
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organization's preferences, then content originating jfrom cigarette producers and/or their 
affiliates might be removed. 

The user content preferences may be a set intersection, complement, or union of the 
content preferences entered directly by the user and the content preferences controlled by the 

5 other party or organization. 

Referring again to Fig. 5, after the observation program 405 receives the user 
preferences at step 505, the observation program 405 activates replacement rules based on 
the user preferences (step 510). Referring to Fig. 8, the observation program performs a 
procedure 800 for activating replacement rules. Designator strings for content are 

1 0 automatically generated and updated by the observation program (step 805) and saved into 
memory for future access by the observation program or any program that might replace the 
content. Typically, the designator string would be unique for the particular content and 
userlD set. Once the designator strings are generated, a set of replacement rules for that 
designator string are updated or created (step 810). Furthermore, the observation program 

1 5 may generate a user identification (userlD) based on the information entered by the user (step 
815). The userlD is implemented when more than a single user is controlled by the 
observation program. This is particularly relevant when the observation program is 
implemented apart from a user's computer, for example, at a proxy server. In that case, the 
observation program monitors and implements the requests of many users and/or the user's 

20 corresponding political, corporate, or affinity groups. 

As shown in Fig. 9, a designator string 850 may include text 855 associated with the 
content, for example, text in a URL, video, or image. Using the example of Fig. 7, a possible 
substring of the text designator string for the Company A advertisement 702 might be 
"companya". A designator string 850 may include an address 857 associated with the 

25 content if the content includes an address that would redirect the user to another location, for 
example a URL or IP address. Using the example of Fig. 7, if the Company A advertisement 
702 provided a link to another web page with information about a credit card from Company 
A, then a possible URL designator string for Company A advertisement 702 may be 
www.companya.com . 

30 A designator string 850 may include a dimension of content 860 in the displayed 

image. For example, in Fig. 7, a dimension designator string may be 4.6 x 1.1. A designator 

10 
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String 850 may include the content in its entirety 865. If the content is a simple image or 
text, then this is more practical. For the Company A advertisement 702, the content 
designator string would include the whole displayed image. A designator string 850 may 
include an identification 870 preferred by the user. For example, in the example of Fig. 7, 
5 perhaps the user prefers to use an image designator string of an image designator (which is 
included in the advertisement 702) instead of the whole image. For example, the image 
designator string may correspond to a credit card logo for an advertisement originating from 
a credit card company. A designator string 850 may include an alphanumeric code 875 that 
is selected to correspond to the content. For example, the designator string for the Company 
1 0 A advertisement 308 may be the code "63400344". A designator string 850 may include a 

position 880 of the content in the display or in the file. For example, the designator string for 
the Company A advertisement 702 may be a position (4.4, 1.1). A designator string may be 
any combination of the above-described designator strings implemented by string 
concatenation. For example, the designator string for the Company A advertisement 702 
1 5 may contain as a substring a position (4.4, 1.1) aad/or a dimension 4.6 x 1 . 1 . 

Replacement rules are a set of groups of content to be removed, content to be 
inserted, and rules between them. The rules define actions that should take place when 
specific content is detected by the observation program 405 and actions that should take 
place to replace the detected content with other content. The rules are written into a memory 
20 that is either local to the user's computer or remote from the user's computer. In any case the 
rules are accessed by the observation program 405 or any server that would be involved in 
the replacement of content. 

Rules take the form of a logic sentence, for example, 

"If contentl is detected and if userlD = 33944, remove contentl 
25 and insert designator string45 1 

"If contentl is detected, if content3 is detected, and if userlD = 
52939, remove contentl and insert designator string238." 
"If address65 is detected and if userlD = 16620, remove content 
from address65 and insert designator string887." 
30 "If originator499 is detected and if userlD = 99232, remove 

content from originator499 and insert designator string333." 

11 
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"If designator string = designator string453 and if userlD 31 109, 
insert content 656." 

Rules have one general form that indicates if a predetermined content for a particular user is 
detected, remove content from application program and replace content with its 
5 corresponding designator string. Rules have another general form that indicates if a 

particular designator string for a particular user is detected, insert other content. Rules may 
also include branch conditionals and/or statements that direct execution of the observation 
program. 

Referring again to Fig. 5, the observation program 405 determines whether a 
1 0 predetermined content is detected (step 515). The observation program 405 monitors for the 
predetermined content in a communication channel between the apphcation program 440 and 
the operating system 435, between the operating system 435 and the server 420, between 
J^I different files accessed on the network 425 by the operating system 435, between the server 

Ul 420 and the files on the network, at the operating system 435, or at the application program 

^ 15 440. 

m In the case of monitoring content in an Internet connection, the observation program 

""I 405 may intercept TCP/IP protocol packets and act as the intermediary between the Internet 

r I and the user ' s computer 415. The observation program would then determine whether the 

fjj TCP/IP protocol packets, after sufficient sequencing and reassembly, include predetermined 

ill 20 content. 

CI The observation program 405 may recognize content using visual, semantic, lexical, 

or any combination of visual, lexical, and semantic techniques. Thus, the observation 
program may use one technique to recognize visual content and another technique to 
recognize lexical content. 

25 In detecting visual images, the observation program may extract an identifier x from 

the visual image -- such an identifier x could be a sequence of bits from the visual image. 
The observation program could, using a transform function H, transform this identifier x into 
an address H(x) that depends on the identifier x. Preferably, such a transformation H(x) 
would be relatively efficient to compute and would resuh in a unique address H(x). If the 

30 detected address H(x) matches more than one of a set of predetermined addresses (each 

address corresponding to predetermined content to be blocked), then further information is 
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required. In this case, the entire or some part of the predetermined content may be accessed. 
However, most likely the detected address H(x) would either match none of the 
predetermined addresses or match one of the predetermined addresses. If the detected 
address H(x) matches none of the predetermined addresses, the observation program 
5 continues to determine if a predetermined content is recognized (step 515). 

The transform function H could be one of several different functions, and selection of 
the particular transform function may be based on factors such as content detection efficiency 
or content detection speed. One example of a possible transform function H is a mid-square 
transfer fimction. The mid-square transfer function squares the identifier x obtained from the 

1 0 analyzed content and then extracts an appropriate number of bits from the middle of the 

squared value to obtain a final address H(x). The choice of the number of bits exfracted from 
the middle of the squared value can be chosen to be any number - the greater the number of 
bits, the greater the chance that the resultant final address H(x) is unique to that content. 
Another example of a transform fimction H is a division fransfer function that is equal to the 

1 5 remainder of the identifier x divided by some number M. 

The fransfomi function H could be a folding transform function that partitions the 
identifier mto several parts, each part bemg of the same length except for the last, adds the 
parts together, and then extracts an arbitrary number from the added parts to obtain the 
address. For example, if the identifier were 1 1010101001 1 1 10, then 1 10+101+010+01 1+110 

20 would yield the address. The observation program can analyze the visual image most 
quickly using the folding fransform function. 

The observation program may use the transform fimction H to detect a predetermined 
content other than visual images. For example a large block of text may be transformed in a 
similar way as a visual image. As known to those skilled in the art, such transform fimctions 

25 described above are quite sensitive to small changes in a bit sequence. If there is a shght 

modification to the original bit sequence or bit sequence errors, an alternative function may 
be selected to mitigate the effect of such random or deliberate changes to the incident bit 
sequence. Other techniques such as content based retrieval, segmentation, or exfraction of 
feature attributes known to those skilled in the art may be appUed instead of the above 

30 referenced fimction. 
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The observation program may use optical character recognition (OCR) techniques to 
detect the predetermined content. In this case, the observation program may recognize 
characters in the predetermined content. During OCR processing, the observation program 
analyzes the image for hght and dark areas in order to identify each alphabetic letter or digit. 
5 When a character is recognized, it may be converted into an ASCII code. 

The observation program may use a parsing algorithm to recognize lexical 
information. Subsequent to OCR or visual images, embedded natural language may be 
parsed. The observation program would then break up the content (for example, the nouns 
(objects), verbs (methods), and their attributes or options) into parts that can then be managed 

1 0 by other programming. The observation program may also check to see that all input has 
been provided that is necessary. 

The observation program may recognize images using a database lookup technique. 
Thus, previously identified content may be stored into a database and the observation 
program would search this database using any of the well known searching techniques, such 

1 5 as index key lookup, OCR, or computer vision algorithms. In index key lookup, an index 
key generated for the image is searched for in an index database. Furthermore, the 
observation program may recognize some content based on other similarly recognized 
content. Another example of a database lookup technique is a training image technique. In 
this technique, a detected image may be classified as being similar or very different from a 

20 set of training images. For example, one may compile a large number of images of an object 
hke the White House. Any detected image may then be classified as either "containing the 
White House" or "not containing the White House". The observation program may take into 
accoimt the position or relative position of the content on a given page for recognition 
purposes. 

25 As described above, the observation program 405 may, based on the received user 

preferences, search for predetermined content in a text string of the URL 630, in an IP 
address corresponding to the URL 625, in the originator of content 620, or in the content 
itself 615. All of the recognition techniques discussed above may be used to detect 
predetermined content in any of these forms. For example, a string comparison may be used 

30 to detect a text string or substring of the URL or an IP address. 
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Since the URL frequently indicates the originator of content through the domain 
name, the text in the URL is often useful when recognizing content. Additionally, if only the 
EP address is provided, then name server lookup of the domain may be necessary to 
determine an originator of content. Within the content itself, OCR and/or logo recognition 
5 may be employed to detect a brand associated with the content. It should be noted that an 
originator of content may or may not be associated with the brand associated with the 
content. For example, GTE may be an originator of an advertisement for a product from 
IBM. Moreover, the service provider or the content provider may be the originator of 
content. 

f 0 If the observation program recognizes content as predetermined content (step 5 1 5), 

then the observation program 405 proceeds to remove the recognized predetermined content 
from the file accessed by the user (step 520). 

Once the observation program 405 has targeted and removed the content for 
replacement, the observation program determines the designator string according to the 

1 5 replacement rules and using the userlD and preferences. Thus, the observation program 405 
would then insert the designator string 850 into the file to be received by the user (step 525). 

Referring also to Fig. 10, once the designator string 850 is inserted into the file, a 
replacement server (such as the server 420, the observation program 405, or some other 
server accessed by the computer system 410) may perform a procedure 1000 to replace the 

20 designator string 850 with a replacement content. For example, if the designator string 850 is 
a URL address 857 such as the www.companya.com address used in the example of Fig. 7, 
then the replacement server may replace that URL address with another address such as 
www.companyb.com and insert an advertisement link 1 100 for a Company B credit card, as 
shown in Fig. 1 1. As another example, if the designator string 850 for the Company A 

25 advertisement is an alphanumeric code, then the replacement server may replace the 
alphanumeric code with a Company B credit card advertisement, as shown in Fig. 11. 

The replacement server first detects the presence of the designator string (step 1005) 
using any visual or lexical techniques that are described above with respect to detecting the 
predetermined content. For example, the replacement server could search the document for a 

30 code that would indicate the removed content and/or the content to be inserted in place of the 
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removed content. Then, using the replacement rules, the replacement server inserts a 
replacement content (step 1010), 

The replacement content can be cached on the user's computer or it can be read over 
the network by the replacement server. The replacement content may be white space -- that 
5 is, the replacement server may simply remove the designator string. Or, in the case of 

graphical or visual information, the replacement content could be white space of a specified 
dimension. The replacement content could be a decorative image of a specified dimension. 

The techniques for content replacement detailed above may be used by a consumer 
wishing to remove all content jfrom a particular source, and in effect boycotting all content 

10 from that source. For example, if the consumer wishes to boycott a company that sells fur 
coats, the consumer could use the techniques for content replacement to remove all content 
coming fi-om that company. 

The techniques, methods, and systems described here may find apphcabihty in any 
computing or processing environment in which electronic content may be viewed, accessed 

1 5 or otherwise manipulated. 

Various implementations of the systems and techniques described here may be 
realized in digital electronic circuitry, or in computer hardware, firmware, software, or in 
combinations thereof A system or other apparatus that uses one or more of the techniques 
and methods described here may be implemented as a computer-readable storage medium, 

20 configured with a computer program, where the storage medium so configured causes a 
computer system to operate on input and/or generate output in a specific and predefined 
manner. Such a computer system may include one or more programmable processors that 
receive data and instructions fi*om, and transmit data and instructions to, a data storage 
system, and suitable input and output devices, 

25 Each computer program may be implemented in a high-level procedural or object- 

oriented programming language, or in assembly or machine language if desired; and in any 
case, the language may be a compiled or interpreted language. Suitable processors include, 
by way of example, both general and special purpose microprocessors. 

Generally, a processor will receive instructions and data fi-om a read-only memory 

30 and/or a random access memory. Storage devices suitable for tangibly embodying computer 
program instructions and data include all forms of non- volatile memory, including 

16 



Attorney Docket No. 11423-002001 



semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; 
magnetic disks such as intemal hard disks and removable disks; magneto-optical disks; and 
CD-ROM disks. 

Any of the foregoing may be supplemented by, or implemented in, specially-designed 
5 ASICs (application-specific integrated circuits). 

A number of embodiments of the present invention have been described. 
Nevertheless, it will be understood that various modifications may be made without 
departing fi-om the spirit and scope of the invention. Accordingly, other embodiments are 
within the scope of the following claims. 
1 0 For example, as shown in Fig. 12, the observation program 405 may perform a 

procedure 1200 to directly insert alternative predetermined content into the file without the 
use of a designator string. Examples of replacement rules for this embodiment are: 
"If address65 is detected and if userDD = 00021, remove content 
from address65 and insert content484." 
15 "If originator499 is detected and if userlD = 99232, remove 

content fi:om originator499 and insert content fi*om originators." 
In the procedure of Fig. 12, the observation program 405 receives user 
preferences (step 1205), activates replacement rules (step 1210), determines if 
predetermined content is present (step 1215), and deletes the predetermined 
20 content (step 1220). Then the observation program inserts new content into the 
location of the deleted content (step 1225). 

The observation program 405 may gather statistical information relating to 
content accessed by the user. This statistical information may be user or affinity 
group specific. The statistical information may then be published or offered for 
25 sale to interested organizations. 

The system 400 may be used by a content provider that is supported by a 
service provider to replace a certain percentage of advertisements with 
advertisement relating to the service provider. In exchange, the service provider 
agrees to accept such advertising as a form of payment fi-om the content provider 
30 for access to network links such as Tl or T3 lines, DSL, or cable modems. 
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The system 400 may be used to insert a hyperlink for a term in a 
document. For example, if a document accessed by the user includes the word 
"sports", the observation program 405 could insert or append to the word a 
hyperlink to content relating to sports, for example a hyperlink to the CNN sports 
section. Thus, whenever the word "sports" appears in a web page, the observation 
program 405 would append or insert the hyperlink wwwxnnxom/cnnsi . The 
observation program 405 would insert such hyperlinks based on the user 
preferences. For example, if the user indicated an interest in sports or an interest 
in sports from CNN, then the observation program 405 would create a 
replacement rule such as "if 'sports' is detected and if userlD = 54232, add link to 
www . cnn. com/cnnsi . 

What is claimed is: 
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1 LA computer-implemented method of controlling content, the method 

2 comprising: 

3 receiving input jfrom a user; 

4 monitoring content in an apphcation program or content exchanged between the 

5 application program and an operating system residing at the user's computer system; 

6 determining whether monitored content corresponds to a predetermined 

7 advertisement; and 

8 if the monitored content corresponds to the predetermined advertisement, replacing 

9 the predetermined advertisement with another predetermined advertisement based on the user 
10 input. 

1 2. The method of claim 1 , wherein receiving input from the user comprises 

2 receiving information about content that the user wishes to control . 

1 3 . The method of claim 2, further comprising generating the other predetermined 

2 advertisement based on the received user information. 

1 4. The method of claim 1 , wherein monitoring content comprises monitoring 

2 content in a database manager, a spreadsheet, a communications package, a graphics 

3 program, a word processor, or a network browser. 

1 5 , The method of claim 1 , wherein receiving input from the user comprises 

2 receiving information from an organization designated by the user. 

1 6. The method of claim 5, ftirther comprising generating the other predetermined 

2 advertisement based on received organization information. 

1 7. The method of claim 1 , wherein monitoring content comprises monitoring 

2 text, video, audio, image, animation, or document link in the apphcation program. 

1 8 . The method of claim 1 , wherein monitoring content comprises monitoring 

2 information related to the originator of content. 

1 9. The method of claim 1, wherein monitoring content comprises monitoring 

2 content exchanged between a network browser and a network server. 

19 



Attorney Docket No. 11423-002001 



1 1 0. The method of claim 1 , further comprising activating rules including 

2 generating rules using the received user input and updating rales using the received user 

3 input and the generated rales. 

1 11. The method of claim 1 0, wherein determining whether monitored content 

2 corresponds to a predetermined advertisement comprises: 

3 extracting an identifier fi-om the content; and 

4 using the rales comparing the extracted identifier with a predetermined set of 

5 identifiers that correspond to a predetermined set of advertisements. 

1 12. The method of claim 1 1, wherein if the extracted identifier matches one of the 

2 predetermined sets of identifiers, replacing the corresponding predetermined advertisement 

3 with the other predetermined advertisement. 

1 13, The method of claim 1 1 , wherein replacing the predetermined advertisement 

2 with the other predetermined advertisement comprises: 

3 replacing the corresponding predetermined advertisement with a 

4 representation of the predetermined advertisement; 

5 detecting the representation; and 

6 replacing the representation with the other predetermined advertisement. 

1 14. The method of claim 1 , wherein replacing the predetermined advertisement 

2 with another predetermined advertisement comprises replacing the predetermined 

3 advertisement with text, video, audio, image, animation, or a link to a document. 

1 15. The method of claim 1 , wherein replacing the predetermined advertisement 

2 with another predetermined advertisement comprises accessing the other predetermined 

3 advertisement firom a user's computer or from a network server. 

1 16. The method of claim 1 , wherein monitoring occurs at the user ' s computer 

2 system. 

1 17. The method of claim 1 , wherein monitoring occurs at a network server 

2 separate from the user's computer. 
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1 18. Software in a computer-readable medium comprising instructions for causing 

2 a computer system to perform the following operations: 

3 receive input fjcom a user of a computer system; 

4 monitor content in an application program or content exchanged between an 

5 operating system residing at the user's computer and the apphcation program; 

6 determine whether monitored content corresponds to a predetermined advertisement; 

7 and 

8 if the monitored content corresponds to the predetermined advertisement, replace the 

9 predetermined advertisement with another predetermined advertisement based on the user's 
10 input. 

1 1 9. A computer system for controlling content, the system comprising: 

2 a client computer programmed to receive input from a user and to operate an 

3 application program; and 

4 a second computer intercoimected with the client computer by an internetwork and 

5 programmed to: 

6 observe content in the application program or content exchanged between the 

7 application program and an operating system residing at the client computer, 

8 determine whether observed content corresponds to a predetermined 

9 advertisement, and 

10 if the observed content corresponds to the predetermined advertisement, 

1 1 replace the predetermined advertisement with a predetermined identifier that corresponds to 

1 2 another predetermined advertisement based on the user input. 

1 20. The computer system of claim 1 9, further comprising a second set of 

2 instructions for programming the second computer to: 

3 receive the predetermined identifier; and 

4 replace the predetermined identifier with the corresponding other 

5 predetermined advertisement. 

1 2 1 . A computer-implemented method of controlling content in an apphcation 

2 program or in a communication charmel between an operating system residing at a user's 

3 computer system and the apphcation program, the method comprising: 
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4 receiving an input from the user; 



5 



observing content exchanged between the apphcation program and the operating 



6 system; 



7 determining whether observed content would direct the user to a predetermined 

8 advertisement; and 

9 if the observed content would direct the user to the predetermined advertisement, 

1 0 replacing the observed content with other content that directs the user to another 

1 1 predetermined advertisement based on the user input. 

1 22. Software in a computer-readable medium comprising instructions for causing 

2 a computer system to perform the following operations: 

3 receive input from a user of a computer system; 

£1 4 monitor content in an apphcation program or content exchanged between an 

5 operating system residing at the user's computer and the application program; 

6 determine whether monitored content would direct the user to a predetermined 

7 advertisement; and 

8 if the monitored content would direct the user to the predetermined advertisement, 

15 9 replace the monitored content with other content based on the user ' s input. 

r I 

111 

ill 1 23 . A computer system for controlling content, the system comprising: 

jj; 2 a client computer programmed to receive input from a user and to operate an 

CI 3 ^plication program; 

4 a second computer interconnected with the client computer by an internetwork and 

5 programmed to: 

6 observe content in the application program or content exchanged between an 

7 operating system residing at the client computer and the application program, 

8 determine whether observed content would direct the user to a predetermined 

9 advertisement, and 

^ 0 if the observed content would direct the user to the predetermined 

1 1 advertisement, replace the observed content with other content based on the user input. 

1 24. A method of filtering content relating to an originator of content, the method 

2 comprising: 
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3 receiving input from a user selecting an arbitrary originator of content; 

4 receiving a request from the user at a computer for an internetwork page; 

5 determining whether the internetwork page includes content relating to the originator; 

6 if the internetwork page includes content relating to the originator, replacing the 

7 related originator content with other predetermined content. 

1 25, The method of claim 24, fiirther comprising sending a message to the 

2 originator of content informing the originator that the content has been filtered by the user. 

1 26. The method of claim 24, wherein receiving input from the user comprises 

2 receiving information about the originator of content. 

1 27. The method of claim 26, ftirther comprising generating the other 

2 predetermined content based on the received user input. 

1 28. The method of claim 24, wherein determining whether the internetwork page 

2 includes content relating to the originator comprises determining whether the internetwork 

3 page includes text, video, audio, image, animation, or document link relating to the 

4 originator. 

1 29 . The method of claim 24, wherein determining whether the internetwork page 

2 includes originator-related content comprises: 

3 extracting an identifier from the content; and 

4 comparing the extracted identifier with a predetermined set of identifiers that 

5 correspond to content relating to the originator. 

1 30. The method of claim 29, wherein replacing the related content with other 

2 predetermined content comprises: 

3 if the extracted identifier matches one of the predetermined identifiers, 

4 replacing the related content with a representation of the related content; 

5 detecting the representation; and 

6 replacing the representation with the other predetermined content. 
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1 31, The method of claim 24, wherein replacing the related content with other 

2 predetermined content comprises replacing the related content with text, video, audio, image, 

3 animation, or document link. 

1 32. The method of claim 24, wherein replacing the related content with other 

2 predetermined content comprises accessing the other predetermined content from a client 

3 computer or from a server computer. 

1 33. Software in a computer-readable medium comprising instructions for causing 

2 a computer system to perform the following operations: 

3 receive input from a user indicating an arbitrary originator of content; 

4 receive a request from the user at a computer for an internetwork page; 

5 determine whether the internetwork page includes content relating to the originator; 

6 and 

7 if the internetwork page includes content relating to the originator, replace the related 

8 content with other predetermined content. 

1 34. A computer system for filtering content that relates to an originator of content, 

2 the system comprising: 

3 a chent computer programmed to receive input from a user and to operate an 

4 internetwork application program; 

5 a server computer interconnected with the chent computer by an intemetwork and 

6 programmed to: 

7 receive input from a user selecting an arbitrary originator of content, 

8 receive a request from the user at the client computer for an intemetwork 

9 page, 

1 0 determine whether the intemetwork page includes content relating to the 

1 1 originator, and 

1 2 if the intemetwork page includes content relating to the originator, replace the 

1 3 related content with other predetermined content. 
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ABSTRACT 

A computer-implemented method of controlling content includes receiving an input 
from the user and monitoring content in an application program or content exchanged 
between the application program and an operating system residing at the user's computer 
system. The method includes determining whether monitored content corresponds to a 
predetermined advertisement. If the monitored content corresponds to the predetermined 
advertisement, the predetermined advertisement is replaced with another predetermined 
advertisement based on the user input. 
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